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What is GSDO? 


• Ground Systems Development and Operations (GSDO) Program 


1 of 3 NASA Programs based at Kennedy Space Center (KSC) 

Established to develop and use the complex equipment required to safely 
handle rockets and spacecraft during assembly, transport, and launch 



Mission: 


To prepare KSC to process and launch the 
next generation of rockets and spacecraft in 
support of NASA’s exploration objectives by 
developing the necessary ground systems, 
infrastructure, and operational approaches. 
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What is RMA? 


• RMA is the acronym for Reliability, Maintainability, and Availability 

- Reliability (R) 

• The probability (likelihood) that a component or system will perform its intended 
function with no failures for a given period of time (mission time) when used under 
specific operating conditions (test environment or operating environment) 

- Maintainability (M) 

• The probability a failed item will be restored or repaired to a specified condition 
within a given period of time 

- Availability (A) 

• The probability that a repairable system will perform its intended function at a given 
point in time or over a specified period of time when operated and maintained in a 
prescribed manner. Thus , availability is a function of reliability and maintainability 


• If “R” is enough, then no need for “M” and “A”; if “R” is not enough, then 
“R” and “M” is needed to make “A” 


f(R,M) = A 


SAIC. A 1 : rights reserved. 
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RMA Analysis Purpose 


• To reduce lifecycle cost by: 

Efficiently and effectively identifying limitations within a system that may cause a 
failure before the intended lifetime 

- Identify unreliable systems that may pose a safety or health hazard 

- Providing specific Reliability requirements for component procurement 

To identify wasted efforts and hardware that were intended to improve Availability, 
but are providing little value 

• To study, characterize, measure, and analyze the failure and repair of systems 
in order to: 

- Improve their operational use by increasing their design life 

- Eliminate or reduce the likelihood of failures and safety risks 

- Reduce downtime (maintenance), thereby increasing available operating time 


~ SAIC. Ai| ng'its reserved. 
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RMA Design Life Analysis Process 


• Ideally, the Reliability Engineering process looks like this: 



Engineering 


WHO 


Success Space 


Failure Space 


PRA 


5H< 


FFBD 


RBDA 


■FMEA 


FTA 


Safety & Mission Assurance 


10Q% 


Start FFBD = Functional Flow Block Diagram 

RBDA= Reliability Block Diagram Analysis 
FMEA= Failure Mode & Effects Analysis 
FTA = Fault Tree Analysis 
PRA= Probabilistic Risk Assessment 


Finish 


Source: Tim Adams KSC-NE 


SAiC.com 


(•TSAIC. All rights reserved. 


SAIL 


EFFORT 




GSDO RMA: Solution to a Challenge 


* GSDO Program needs to deliver high launch probability 

- Lunar missions and beyond require multiple launches and payloads to achieve mission 
goals 

Commercial, DoD, and NASA customers will desire high availability from Ground Systems 
for launch support 

- The cost of each launch "scrub" is severe 

• De-tanking vehicles, re-synchronizing orbits, rescheduling Range conflicts, resting crew, etc 

• If a GROUND SYSTEM causes this scrub ( when the vehicle was otherwise "Available"), then the 
community's penalty is even more severe 

• High Probability of Successful launch is needed; however, challenges were 
faced: 

KSC Ground Systems delivered 88% probability of launch during Space Shuttle Program 
(SSP) 

- KSC Ground Systems Constellation Program (CxP) requirement was 99% probability of 
launch for the last 10 hours of launch countdown 

GSDO Program requirement is 98% Inherent Launch Availability for any given launch 
countdown 
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GSDO RMA: Solution to a Challenge 


Risk Factors for GSDO: 


GSDO 


Ground 

Systems 

Human 

Errors 

Range 

Conflicts 

Range 

Systems 


Range 

Intrusions 


Hurricanes 



1 


Risks for launch probability: 

- GSDO 

Space Launch System (SLS) - 
the Launch Vehicle 

Orion Multi-Purpose Crewed 
Vehicle (MPCV) 

Environments - Launch Weather 
(Wx), Abort Wx, Sea State 



Example of Probability of Architecture on Any Given Launch Attempt* 

*Does not necessarily represent actual risk probabilities. 
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GSDO RMA: Requirement Development 


GSDO 



Needed to put requirements in place to minimize risks to 
successful launch support 

- Only could control risks to Ground Systems 
design and upgrades 

- Allocated Availability requirements to 
Ground Systems 

* Inherent Launch Availability 

• Operational Availability* 



GSDO Launch Availability 

Availability of XXX% for 30 days 


Human 

Errors 


Range 

Conflicts 


Range 

Systems 


Range 

Intrusions 


Hurricanes 


GSDO Operational Availability 
Shall achieve an operational availability (Ao) 
of at least 80% (TBR) from the start of launch 
countdown up to 14 days after a launch 
scrub. 


GSDO Inherent Launch Availability 
Shall have an inherent launch 
availability of not less than 98% (TBR) 
for any single launch attempt. 


Human Errors 
Range Conflicts 
Range Systems 
Range Intrusions 
Hurricanes 


*The Operational Availability requirement is not the classical Operational Availability (AJ calculation. It is allocated as 
Inherent Availability (A). From a system design point of view, the A, is of more interest than A 0 because spares and repair 
capability involve resources and trade-offs external to the system design. ^ m m 
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Operational availability cannot be controlled by system design, but Inherent Availability can 
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GSDO RMA: Requirement Allocation 


• Reliability allocations made via improved Reliability Apportionment 
Method 

- Accounts for knowledge of Ground System performance , design , and 
i/se 


Maintainability allocations made via an improved MIL-HDBK-417A 
method 

- Accounts for knowledge of Ground System design , fault isolation 
techniques , and maintenance design characteristics , i.e., accessibility 
on the pad ^ 


VO .98* « 0.999713- 


-< 


>- 



i*l AvailaMHty 
0.80 


^■1 


\fTTf{. * 37 hrs 


^0.369 =0.920278- 


M7TR ss =74hrs- 


1 ss 

0.90 
57 hrs 


1 SS 
0.95 
33 hrs 




4 SS 
0.90 
85 hrs 


1 SS 
0.95 
85 hrs 




2 SS 


1 SS 


82 hrs 


82 hrs 
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GSDO RMA: Requirement Allocation 


• “Management reserve” is built into each RMA requirement allowing for 
room for growth in GSDO subsystems 

- Fraction of the overall requirement is 
unallocated 

- If the cost for an availability improvement 
in a subsystem design outweighs the 
benefit in increased GSDO launch 
availability, there is enough management 
reserve to leave the design as-is, in most 
cases 

• The RMA analyses are completed in parallel with the design and upgrade 
schedules 

RMA analysis is a required product for design milestones (30/60/90 or 45/90) 

- RMA analyses are performed as requested to assist in trade studies 



£'■ SAIC Al! rights r “i-e’ r V9d 
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GSDO RMA: Requirement Allocation 


• Allocation is an iterative process 


12 




As designs are analyzed, allocations may need to be adjusted 

Allocate Availability Requirement 
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RMA Analysis of GSDO Subsystems 


1 ) Develop Component List 
from Subsystem Drawings 



fc 7, ~ 11 ■■■ i , e#g i i i Q ej n ta 




2) Develop Reliability Block 
Diagram (RBD) from 
Subsystem Drawings 
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Cut Set Analysis 


3) Determine Reliability, 
Maintainability, and 
Availability from RBD 


4) Determine Cut Sets 
from RBD 
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5) Determine Importance 
Measures from Cut Sets 


Importance Measures 


6) Report Results and 
Recommendations in 
Subsystem RMA Reports 
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RMA Analysis of GSDO Subsystems 


• Example of component data using COTS software: 
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RMA Analysis of GSDO Subsystems 


• Analysis of components will sometimes include Weibull analysis to 
attempt to determine what type of failures are experienced 

- Used for similar components 

- Used for heritage subsystems to characterize failure types seen: 

• Early failures (burn-in) 

• Useful Life (random failures) 

• WearOUt The Bathtub Cum 


Probability 
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RMA Analysis of GSDO Subsystems 


• Reliability Block Diagram (RBD) Analysis (RBDA): 

Predicts reliability (uptime), maintainability (downtime), and availability 
(mission readiness being a function of uptime and downtime) 

The RBDA method is used to estimate and analyze the reliability and 


availability for the systems containing at least two or more elements 
* RBDA is a “top-down” method in success space 



© SAIC. All rights reserved. 
















RMA Analysis of GSDO Subsystems 


• RMA Team converts each drawing (mechanical and electrical) into a 
Reliability Block Diagram (RBD) 

Verify accuracy and understanding of the components and their connections 
with the design team 
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RMA Analysis of GSDO Subsystems 


• RMA team determines the RMA of the subsystem by using both analytical and 
Monte-Carlo simulation calculations with at least 1,000,000 iterations. 

Confidence Level set at 95% 

• Compare results to requirement 

RMA Requirements Simulation Results 


Reliability 

(for24hrs) 

Maintainability 

(hrs) 

Availability 

(A,„ h ) 

0.99900 

48 

0.998017 


Reliability 
(for 24 hrs) 

Maintainability 

(hrs) 

Availability 

(A,„ h ) 

0.998448 

12.59 

0.999529 


• Perform sensitivity analysis to verify consistency in simulations 

Different random number seeds for Monte-Carlo simulations 


Random 

Seed 

Reliability 

Failures per 
Million 

MTTR 

(hrs) 

Availability 

Lower Bound 

Point 

Estimate 

Upper 

Bound 

Lower Bound 

Point Estimate 

Upper 

Bound 

1 

0.998564 

0.998636 

0.998708 

54.07 

12.59 

0.999521 

0.999562 

0.999603 

10 

0.998448 

0.998523 

0.998598 

67.10 

12.61 

0.999439 

0.999484 

0.999529 

100 

0.998557 

0.998630 

0.998703 

58.08 

12.63 

0.999520 

0.999561 

0.999602 
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RMA Analysis of GSDO Subsystems 


RMA Team performs Cut Set Analysis (CSA) 

- Set of basic events [failures] where the joint occurrence of these basic events results in 
the failure of the system. 

• Minimal cut set is a set that “cannot be reduced without losing its status as a cut set” 

Provides clear indication of where most likely failure paths would be depending on the 
accuracy of the RBD and the accuracy of the failure data of the components 



Cut Set 


Minimal Cut Set 


fCttVt TVC-I 
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RMA Analysis of GSDO Subsystems 
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When a small number of failure paths make such large contributions to 
subsystem unavailability, isolating the key failure paths becomes obvious 


This enables the design team to focus 
on either: 



Improving the design to correct the high 
failure nodes (improving reliability), or 


- Ensuring that the component is able to 
be repaired to an operational state as 
quickly as possible (improving maintainability) 
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RMA Analysis of GSDO Subsystems 


• Example of Cut Set Data 


CUM UNAVAIL % 

UNAVAIL % 

UNAVAIL 

Component Failure 

Component Failure 

5.92% 

5.92% 

0.000146 

ICPSU Position Encoder 1 


11.83% 

5.92% 

0.000146 

ICPSU Position Encoder 3 


13.13% 

1.30% 

3.22E-05 

4000 PSI Hydraulic Supply Pressure 
Xducer (36583MT-1) 


14.22% 

1.08% 

2.68E-05 

2200 PSI GN2 Supply Valve 
(36583A12) 


15.17% 

0.95% 

2.36E-05 

Fully Extend Switch (36596) 


15.97% 

0.80% 

1.97E-05 

Slow Extend/Retrack Valve (36583A9) 


16.67% 

0.70% 

1.72742E-05 

CB (GSP to RIO) 50A - Side A 

CB (GSP to RIO) 50A - Side B 

17.37% 

0.70% 

1.72742E-05 

CB (GSP to RIO) 50A - Side A 

Circuit Breaker (CB) 2A - Side B 

18.07% 

0.70% 

1.72742E-05 

CB (GSP to RIO) 50A - Side A 

Circuit Breaker (CB) to TB - Side B 

18.77% 

0.70% 

1.72742E-05 

CB (GSP to RIO) 50A - Side A 

Circuit Breaker to Chassis - Side B 

19.47% 

0.70% 

1.72742E-05 

CB (GSP to RIO) 50A - Side A 

Circuit Breaker to Ind & Enable - 
Side B 

20.17% 

0.70% 

1.72742E-05 

Circuit Breaker (CB) 2A - Side A 

CB (GSP to RIO) 50A - Side B 

20.87% 

0.70% 

1.72742E-05 

Circuit Breaker (CB) 2A - Side A 

Circuit Breaker (CB) 2A - Side B 
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RMA Analysis of GSDO Subsystems 


• GSDO RMA Team performs Importance Measure Analysis (IMA) 

Assesses the importance of the components in the subsystem or the sensitivity of the subsystem 
RMA to changes in the components’ failure rates 

- Quantify the criticality of a particular component within a system design. 

- Used as tool for identifying system weakness, and to prioritize RMA improvement activities 

Change in the failure rates of the components (or adding redundancy to account for the high failure 
rate) with the highest importance measure percent contribution will have the most significant effect 
on increasing subsystem reliability 


This unique method described in paper written by RMA team, entitled, “ Comparison 
Modeling of System Reliability for Future NASA projects ” and presented at International 
Reliability and Maintainability Symposium (RAMS) in January 2012 


Flex Duct 2 26% ^Electrical Connector. 


Bectncd Line. 4 79%. 




1. Ensure this component/LRU 
is on hand in order to repair 
and/or replace when failed. 

2. Ensure personnel are trained 
in procedures for repair 
and/or replace. 

3. Ensure procedures are 
optimized for repair and/or 
replace. 


I I § * 2 3 | S 3 

■ CBTV ■ FCSSC ■ FCSBSC UCBSC 
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RMA Analysis of GSDO Subsystems 


• GSDO RMA Team reports results and recommendations in Subsystem 
RMA reports 


Example recommendations for RMA improvement: 


• Have redundant components on separate busses 

Improved availability by an order of magnitude (0.995 to 0.9994) 


* Move control and monitoring to different Programmable Logic Controller (PLC) 
Had redundant monitoring on same PLC (see next page) 

Improved availability by three orders of magnitude (0.9993 to 0.999999) 


Example recommendations for trades: 

• Tertiary power system provides little to no 
improvement in availability (0.999995 to 
0.999996); does not justify additional 
weight, space, and cost 

• Avionics architectures: triplex voter 
improves availability, however, self-checking 
pair does not 
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RMA Analysis of GSDO Subsystems 
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R(10) = 0.999300 


R(10) = 0.999999 
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RMA Analysis of GSDO Subsystems 


• Effectively monitoring and tracking RMA analysis results for 
management 

Management informed of risk to achieving requirements almost immediately 

• RMA tracking & reporting methodology effective and efficient in 
communicating recommendations for RMA improvements 

Can quantify RMA improvements versus cost, scheduling, weight, space 


impacts 
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RMA Analysis of GSDO Subsystems - Summary 


• GSDO requirements allocated to subsystems 

Inherent Launch Availability is allocated to those subsystems in the launch 
countdown window 

Operational Availability is allocated to those subsystems not included in the 
Inherent Launch Availability allocation, but needed in the event of a launch 
scrub 


• GSDO RMA team performing RMA analysis of subsystem designs and 
upgrades, as well as heritage subsystems 

Analysis of heritage subsystems includes Weibull analysis to attempt to 
determine what type of failures experienced: Early failures (burn-in), Useful Life 
(random failures), Wearout 

• GSDO is tracking and reporting RMA analysis results of ground hardware 
and software components 


26 


FTA, FMEA, etc are performed as part of the design development cycle to 
drive out subsystem hazards and single failure points 
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Other RMA Analyses 


• Failure Modes & Effects Analysis (FMEA) 

• Fault Tree Analysis (FTA) 

• Probabilistic Risk Assessment (PRA) 

• Historical Component Failure Rate Determination 

• Component Burn-in and Test Time Requirements 
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Failure Modes and Effects Analysis (FMEA) 


• Inductive (bottom-up) method where a table that describes the way or modes in which each 
system component can fail and assess the consequences of each of these failures is 
generated 

• Determines hardware criticality 

• Identifies failure modes that do not meet applicable Program reliability requirements 

• Identifies the potential for single point failures 

• Identifies areas where the design does not meet the failure tolerance requirements 

• Changed from qualitative to quantitative by assigning values to: 



Probability of the failure occurring, 

Severity of the effect of the failure on 
the operation of the systems, 

J Rating 

Description 

Criteria 

2 . 

i 

Very Low or None 

Minor nuisance 

3 . 

Probability that the system controls will 
detect and eliminate the failure before the 

2 

Low or Minor 

Product operable at 
reduced performance 

design is complete. 

The product of all three values is the risk 

3 

Moderate or 
Significant 

Gradual performance 
degradation 

priority number (rpn) 

4 

High 

Loss of function 



5 

Very High or 
Catastrophic 

Safety-related 
catastrophic failures 
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Fault Tree Analysis (FTA) 


• Deductive (top-down) method that generates a symbolic logic model that traces 
and analyzes the failure paths from a predetermined, undesirable condition or 
event (called the top event) of a system to the failures or faults 


Can be qualitative or quantitative - we do quantitative 

FTA is an event-oriented analysis in contrast to the RBD, which is a structural- 
oriented analysis 
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Probabilistic Risk Assessment (PRA) 


• Systematic and comprehensive methodology to evaluate risks associated with a 
complex system 

• Risk in PRA is defined as scenarios, associated frequencies, and associated 
consequences 

Risk management involves prevention of adverse scenarios and promotion of favorable 
scenarios 

- NASA uses Risk metrics of probability of loss of vehicle, mission failure, etc 

• Goal is to describe how the system and its elements respond to an undesired 
initiating event, such as lightening or fire 

• Quantitative 

- Magnitude of the possible adverse consequence 
Probability of the occurrence of each consequence 

• Include: 

- Human Reliability Analysis (HRA) 

- Common-Cause-Failure Analysis (CCF) 
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Probabilistic Risk Assessment (PRA) 
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Historical Component Failure Rate Determination 



0.10 - 


FT081? 


n: 2991 8171 
p: 0.9761 
p» 0.9628 


Probability 

VJ-UNE 


• Using 442 PRACA records: 

- Input into Weibull Analysis 

- Results: 

• It is in its useful life cycle, with 
random failures 

• p=1.0615 

• MTBF = 2991 hrs 

- Assumptions 

• Repair Time: 223 hrs 

• Inspection Time: 8 hrs 

- Maintainability 

• Maximum Availability = 87% 

• Inspection Time = 220 hrs 


1 10 100 1000 10000 
Time 
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Component Burn-in & Test Time Requirements 


• RMA Analysis can determine product testing parameters 

Reliability life testing can quantify reliability or safety goals 
Burn-in test times can determine constant failure rates 
Can determine acceptance test parameters 

• The Weibull shape parameter (p) corresponds to the different failure modes for 
components 

Infant mortality when p is less than 1 
- Random defects when p is equal to 1 
Wear-out when p is greater than 1 

• The results of System Reliability 
analysis can be misleading if 
components are not properly 
up-screened (burned-in) or used 
under a certain bias condition 
where different failure modes may 
occur 

<S SAIC. A!' rights reserved. 
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Why Have RMA Analysis in Design Process? 


• RMA Analysis provides quantitative results, which can be used to justify 
component replacement, system upgrades, cost effectiveness of “abandon 
in place” concepts for systems, etc 

• GSDO RMA process allows for verification and traceability of RMA 
requirements 

• GSDO RMA Analysis encompasses entire design life cycle 

- RBDA - FTA 

- FMEA - PRA 

• RMA Analysis can be used to optimize timeline and launch availability 
results 

Provide MTBF, failure distribution , MTTR, and repair probability to Ground 
System hardware and software 

• RMA Analysis can be used to optimize Logistics considerations 

- Spare parts need - Preventative Maintenance requirements 

Logistic Facility space - Maintenance Personnel Requirements CA IT 
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GSDO RMA Analysis Papers 


• GSDO RMA Team Papers Published: 

“On Component Reliability and System Reliability for Space Missions” IEEE International 
Reliability Physics Symposium (IRPS) 2012, Anaheim, CA, 2012 

- “ Comparison Modeling of System Reliability for Future NASA Projects,” Reliability, 
Availability, and Maintainability Symposium (RAMS) 2012 Conference, Reno, NV, 2012. 

" Constellation Ground Systems Launch Availability Analysis: Enhancing Highly Reliable 
Launch Systems Design,” American Institute of Aeronautics and Astronautics (AIAA) 
2010-2180, SpaceOps 2010 Conference, Huntsville, Alabama, 2010. 

• Future Papers for Reliability and Maintainability Symposium (RAMS) 2013 
(January 2013 in Orlando, FL): 

- “ Allocating Reliability & Maintainability Goals to NASA Ground Systems” 

• Paper described GSDO RMA Allocation process for GSDO 

“ Determining Component Probability from Problem Report Data Used in Ground Systems 
for Manned Space Flight” 

• Describes process of capturing qualitative PRACA failure data for use to determine quantitative 


component reliabilities 
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Thank You 


Amanda M. Gillespie, ASQ CRE, Reliability Engineer 

Mail Code: SAIC-LX-03 1 Kennedy Space Center, FL 32899 

Tel: 321.867.6741 | Email: amanda.gillesDie-l@nasa.gov or amanda.m.gillesDie@saic.com 

Mark W. Monaghan, Ph.D., Reliability Engineer 

Mail Code: SAIC-LX-03 1 Kennedy Space Center, FL 32899 

Tel: 321.867.0484 | Email: mark.w.monaghan@nasa.gov or mark.w.monaghan@saic.com 
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Acronym & Abbreviation List 


KSC-NE 

KSC Design Engineering 



MTBF 


Multi-Purpose Crewed Vehicle 
Mean Time Between Failures 

NASA 

National Aeronautics and Space 
Administration 

PLC 

MM 


PRA 

Probabilistic Risk Assessment 

PTC 

PP 


R&M 

Reliability & Maintainability 

Hi 

l&KM 

symposium 

RBD 

Reliability Block Diagram 

RBDA 


RMA 

Reliability, Maintainability, and Availability 


A* 

Inherent Availability 

|MI 


ASQ 

American Society for Quality 



Cut Set Analysis 

DoD 

pepartment of Defense 

FMEA 

Failure Mode & Effects Analysis 


iilliii! 


GSDO 

Ground Systems Development and 
Operations 

IMA 



KSC 

Kennedy Space Center 
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